Fusing eye-gaze and speech recognition for tracking in an automatic reading tutor - a step in the right direction?
نویسندگان
چکیده
In this paper we present a novel approach for automatically tracking the reading progress using a combination of eye-gaze tracking and speech recognition. The two are fused by first generating word probabilities based on eye-gaze information and then using these probabilities to augment the language model probabilities during speech recognition. Experimental results on a small dataset show that the tracking error rate of the system using only speech recognition is 34.9% whereas the tracking error rate for the system that incorporates eye-gaze tracking into the speech recognizer is 31.2% – a relative improvement of 10.6%.
منابع مشابه
Designing and implementing a system for Automatic recognition of Persian letters by Lip-reading using image processing methods
For many years, speech has been the most natural and efficient means of information exchange for human beings. With the advancement of technology and the prevalence of computer usage, the design and production of speech recognition systems have been considered by researchers. Among this, lip-reading techniques encountered with many challenges for speech recognition, that one of the challenges b...
متن کاملDeveloping a reading tutor: Design and evaluation of dedicated speech recognition and synthesis modules
When a child learns to read, the learning process can be enhanced by significant reading practice with individual support from a tutor. But in reality, the availability of teachers or clinicians is limited, so the additional use of a fully automated reading tutor would be beneficial for the child. This paper discusses our efforts to develop an automated reading tutor for Dutch. First, the dedic...
متن کاملApplication of eye tracking technology for screening and rehabilitation of children with special needs.
Eye tracking is a technology for measuring eye movements when looking at a point. The length of time to look at a particular situation, the sequence of observations, as well as points of attention are processed in the process of eye tracking and eye movements are identified by topics such as fixation, saccade, etc. Information about how the eyes move at a special moment has many potential appli...
متن کاملThe effect of adding gaze direction recognition to stabilizing exercises on pain, muscular endurance and proprioception women with chronic non-specific neck pain
Aims and background: Gaze direction recognition is one of the new treatments method for neck pain. The positive effects of stabilization exercises in various studies on neck pain have also been confirmed. Therefore, the aim of this study was to investigate the effect of adding a gaze direction recognition program to common stabilizing exercises on neck pain intensity, muscular endurance and pro...
متن کاملA New Method for Eye Detection in Color Images
The problem of eye detection in face images is very important for a large number of applications ranging from face recognition to gaze tracking. In this paper we propose a new algorithm for eyes detection. First, the face region is extracted from the image by skin-color information. Second, horizontal projection in image is used to approximate region of the eye be obtained . At last, the eye ce...
متن کامل